Note:Ignore this PR. A lot of it has already been split into separate pull requests. I'll close this when there is nothing left to split.

This is part of service feature parity. It is not ready for review, but feedback is welcome if you can tolerate the chaotic writing below. Once #127 is merged this will need to be rebased onto master (omitting d980051).

Futures are a way to support multiple simultaneous client requests. Coroutines are a way to support futures inside of callbacks in an executor. Futures need a task running in parallel to update the result. A task can get to the executor via

• a callback in the executor calling await future
  • Future.__await__ can invoke the task, and yield if the result is not ready yet.
  • Callback itself must be a coroutine (i.e. async def)
• code outside giving the future to the executor directly
  • Add an equivalent of rclcpp::Executor::spin_until_future_complete(future)
    • In asyncio everything has a reference to the event loop so adding a task is easy, but not everything has a reference to the executor in rclpy. One possibility is a future having a reference to a task that needs to be executed to complete it. spin_until_future_complete could add that task to the executor.

Current state:

• Added executor support for coroutines
  • async def/await only. Does not support generator/yield from coroutines (but it could)
  • executor wait_for_ready_callbacks yields new callbacks first, then old coroutines that yielded the last time they were run.
    • Requires an executor to run the iterator returned from wait_for_ready_callbacks until StopIteration or old coroutines may never get resumed.
    • Maybe for some cases it makes sense to interleave new tasks with old tasks? I think wait_for_ready_callbacks should only yield new tasks, and another method should yield old ones. Custom executors should be responsible for implementing their own old-vs-new task strategy.
  • Service callbacks cannot be coroutines. (not impossible; haven't figured out a clean implementation).
• Executor yields a Task instance
  • Task class hides the difference between a normal function and a coroutine from an executor
  • Task done callbacks get executed in series after the original callback/coroutine finishes
    • Task is greedy and executes as many done callbacks as it can, pausing only if one is a coroutine that yields.
    • asyncio.Task done callbacks get scheduled in the event loop, but this would be a problem in rclpy because it could mean another rcl_wait before executing a done callback.
  • Not using asyncio.Task because it is not thread safe, and it expects a reference to an asyncio event loop
• Does not yet have a Future class
  • asyncio.Future is await-able but not thread safe and expects an event loop reference (though partially works without it)
  • concurrent.futures.Future is thread safe but not await-able (Maybe could be subclassed for this)
  • In asyncio.Task is a subclass of future, so maybe a Task could be it's own future too?

connects to #123